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• For a college or university to grant educational credit for learn- 
ing a person has gained outside the classroom — through work or other 
experiences — the school must first determine whether the learning is 
worthy of college credit:. 

How a school should go about setting the standards against which 
experiential learning is to be evaluated is the topic of this report. 
In it, the authors — 

•discuss the results of a survey of present practices in setting 
standards among colleges and universities; 

•set forth a framework for understanding standard setting; 

•present principles that colleges and universities can use in 
establishing standards for assessing experiential learning. * x 

This report provides a clear overview of standard setting for 
college and university officials who must assure that the standards 
used for assessing experiential learning are appropriate and fair. 
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PREFACE 



Helping colleges and universities design and conduct sound 
experiential learning programs, based *on sound assessment procedures, 
is the key mission of the Council for the Advancement of Experiential 
Learning (CAEL). In their traditional offerings, most institutions of 
higher education pledge a commitment to high quality. Given the concern 
for sound and equitable educs^ional assessment, the higher education 
community must continue to develop and refine standards of acceptable 
performance and achievement for giving academic Recognition for both 
classroom-based and experience-based learning. 

A review of CAEL literature led me to the realization that few CAEL 
aids foi the assessment of experiential learning directly address the 
task of setting standards. Because assessment and the setting of stand- 
ards-are parts of the process of educational measurement, CAEL joined 
forces with the ERIC Clearinghouse on Tests and Measurements to address 
the need for aids in standards development. 

CAEL and the ERIC Clearinghouse, as co-sponsors of this study, asked 
Paul I. Jacobs, director of testing and assessment at Thomas A. Edison 
State College, and Joan Xnapp, a program administrator at the College 
Board Division of the Educational Testing Service, to prepare this 
report. They were assisted by the staff of the' ERIC Clearinghouse and 
its director, Barbara Wildemuth, and by CAEL members who participated 
in a survey of present practices. CAEL institutions were helpful and 
enthusiastic in their responses. We are grateful to John Fremer of the 
Educational Testing Service for his thoughtful review of the report; his 
suggestions and comments led to important revisions. 

Morris Keeton ^ 
President 

Council for the Advancement 
&f Experiential Learning 

«* 

v 
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The Need for Increased Vigilance on Standards 

Most educators would rush to assert that experiential learning — 
learning that takes place outside the classroom — has been part of higher 
education for a long time. Cooperative education,* studio art, clinical- 
internships, and field exercises, for example, are' now deeply rooted and 
widely accepted. Most faculty members, when questioned, affirm that 
standards for" judging learning outcomes from experience-based programs 
are as strong as those for forraai college classes* 

However, doubts and misgivings have resulted from the'^rapid growth 
4nd diversity of experiential programs and the use of experiential 
learning assessment as a* recruiting and marketing device. (Although 
traditional experiential learning programs are equally diverse and more 
numerous and extensive, their longevity and acceptance as common practice 
have protected them from comparable skepticism.) 

The first cooperative education program was formed about 1900. ^By 
1955 there were about 65 programs. Also, the varieties of Experiential 
learning opportunities have increased during the last two^ decades; 
service learning, noncollege learning, and* prior learning are new terras 
in experiential education. , ' , 

/ > / 

The Council for the Advancement of Experiential Learning (CAEL) is 
an excellent barometer of the times. . CAEL started at the Educational 
Testing'^Service in 1974 as a research and development project guided by 
12* institutions. Additional institutions wanted to' participate in uhe 
•project, and as a result of this heightened interest, CAEL became a 
chartered educational association in 1976. CAEL membership in 1981 
includes approximately 300 institutions. 

There has been a corresponding* increase in the number of students 
taking advantage of these new opportunities. In ,1969, approximately 
20,000 students participated in cooperative education. Current estimates 
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point to a cooperative education enrollment of about 170,000, evidence 
of the boom in experiential learning described by Keeton and Tate (1978). 

Along with this boom, there has been a decline in the enrollment 
of traditional-aged students in four-year programs at many colleges and 
universities* Some institutions have attempted to recruit more students 
with the ^enticement of experiential learning programs, which has caused 
some educators to question whether credentialing standards are eroding. 
Alarmists have asked whether credit for experience can lead to a Ph.D. in 
house painting. Even in traditional universities grade inflation has 
been present for a long time; degree inflation may be the derogatory terra 
of the future. 

** 

These matters are of no small concern to accrediting bodies. The 
notion of credit for learning obtained outside a classroom setting 
through a college without physical boundary, is startling to the monitors 
of academic quality. Accrediting bodies that have worked to formulate 
procedures and criteria for promoting sound education in traditional 
college settings are less than comfortable with the "new" programs and 
institutions. 

Such concerns call attention to the need for colleges and univer- 
sities to establish and maintain adequate standards for awarding credit 
for experiential learning. An important distinction must be made between 
assessment and setting standards for assessment.^ Assessment is the 
process by which experiential learning is identified, evaluated, and 
equated with an amount of college credit. Setting, standards for assess- 
raeat refers to the process of establishing the criteria against which 
experiential learning is to be evaluated to determine whether it is 
adequate, worthy of 0 college credit. 

This report seeks to address these concerns by examining what is 
now being done when standards for the assessment of experiential 
learning are set and what should be done in setting standards. The 
study approach was to conduct a mail survey of CAEL members to ascertain 
present practices, review the literature, filter the findings through 
authors 1 experience with assessment, and develop some principles that 



colleges and universities can use in establishing standards for 
assessing experiential learning. 

Throughout this report, readers must bear in mind that there are 
two major types gf experiential learning: sponsored and nonsponsored* 
Sponsored experiential learning occurs under the auspices of a college 
or university where the learner is enrolled * The learning activity is 
part cf the student's program of studies, planned and supervised by the 
faculty at the institution* Sponsored learning programs include: 

Career Exploration * Supervised placement in business , government , 
a service organization, or a profession so that the student can become 
familiar with career possibilities and develop employment-related skills* 

Career or Occupational Development * Placement, chosen by the 
student in consultation with an advisor, so that the student can advance 
job skills and gain experience related to a specific career* 

Cooperative Education * Classroom experience integrated with 
practical work experience, when a minimum standard of successful 
performance on the job is a requirement for the institution's degree* 

Professional Training * Service in assigned responsibilities under 
the supervision of a professional - for example, in education, medicine, 
law, social work, nursing, or the ministry - so that the student can 
apply theories and knowledge learned in the classroom* 

Public Service Internship * Service in an appropriate institution 
for a specified period of time, usually from ten to fifteen weeks* 

Social or Political Action * Placement, under faculty sponsorship, 
that provides an opportunity for students to work for change, through 
community organizing, political activity, research, or action projects* 

Personal Growth and Development * A program in an off-campus setting 
that is designed to further personal growth and development, such as the 
wilderness survival programs* 

Cross-Cultural Experience * Involvement in another culture or 
subculture in the United States or abroad* 
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Experiential learning that is not planned and supervised by a 
college or university or prescribed in a learner's program of studies is 
called nonsponsored learning • This learning occurs most often before 
enrollment. Such learning experiences migjit include: * life accomplish- 
ments (e.g?;* public speaking), work (e.g., computer programming), hobbies 
(e.g., painting, writing, or photography), community service (e.g., 
chairing a charity campaign drive) and self-directed learning projects 
(e.g., learning a foreign language). 



1 
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THE SURVEY OF CAEL MEMBERS 

Polling CAEL member institutions was a direct approach to 
determining the current state of the art in standard setting ♦ These 
colleges and universities assess experiential learning in their programs, 
' and the conversion of learning to credits and the setting of standards 
was assumed to be part of these institutions' assessment procedures* 

Through work in this area during the last eight years, the ar'hors 
knew that institutions use three major procedures for awarding credit 
for experiential learning: 

1 • The student's competence is observed and compared to a predetermined 
standard * 

Data Processing Example: Through a mastery test, the student 
demonstrates (a) the ability to apply a programming language to the 
solution of problems in business and (b) a working knowledge of 
computer logic\and flowcharting. 

Communications Example: The student can give an extemporaneous 
speech that is informative, interesting, and persuasive* 

2. The student's competence is observed and compared with the competen ce 
of others * 

Data Processing Example: * The student's test performance shows 
a level of knowledge about programming approximately the same as 
that shown by students earning C's in an introduction to digital 
programming course. 

Communications Example: The student can give an extemporaneous 
speech of the same quality as that of speeches by students at the 
end of a one-semester public speaking course. 

3. The ptudent*s competence may or may not be observed but is inferred 
from the length and breadth of the student's experience. 
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Data Processing Example: The student has been involved with* 
data processing long enough to have worked with unit record 
equipment, and. early computers. Her experience today includes, the 
- newest equipment and teleprocessing . The student ^ ha$ recently 
, been promoted to head of data processing in a large public school 
system* 

Communications Example: The student has been active in student 
government and has given many speeches to both large and small 
groups while campaigning for office* He has been invited to address 
the annual meeting of the National Student Government Association* 

The last method \ is the least justifiable for awarding credit. 
Because academic recognition should be given for learning from 
experience, not for the experience itself, t^e survey was not expected 
to reveal many standard-setting procedures that\ equate experience per se 

with credit or other academic recognitions However, because in 

\ 

\ 

traditional higher education classroom learning sometimes inferred 
from time spent and then converted to credit, the survey was expected to 
show a few spillover instances of the granting of credit for experience* 



Method and Findings 



The survey was conducted in 1979 to determine the extent of each of 
the three practices. Survey recipients were extremely cooperative; 65 
percent responded. Those who could not answer the questionnaire 
forwarded it to a person at the institution who could. Mpre than 25 
percent of the respondents included attachments describing program 
procedures and policies. r 

In the survey, the institutional representative of each of 
330 CABL institutions was asked (1) to state which of the three methods — 
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comparison to a predetermined standard, comparison to the competence of 
others, or inference from the student's experience — is predominantly used 
_at the inatitution, (2) to state which of the three methods are sometimes 
used as an alternative, and (3) to describe what additional methods, if 
any, are used. A summary of the survey responses is given in Figure 1. 

The responses to the third question, regarding additional methods, 
revealed an important finding. In addition to the responses shown in 
Figure 1, other items that were cited by less than five percent of the 
respondents as methods of standard setting included stadent logs and 
essays, certificates and licenses, oral examinations, product assessment, 
job site ^servations, American Council on Education (ACE) credit recom- 
mendations, self-evaluation, w and documented learning. These responses 
make clear that at several institutions standard-setting practices could 
not be distinguished from assessment modes (such as logs and portfolios). 



Figure 1, Responses to the Survey of CAEL Member Institutions* 

Questiou Percentage of Returns 

1. Dominant Method 

Comparison to a predetermined standard 42 

Comparison to the competence of others 21 

Inference from student's experience 37 

2. Other Methods 

Comparison to a predetermined standard 33 

Comparison to the competence of others 40 

Inference from students experience 27 ; 

3. Additional Methods 

Do not have or are still developing 8 

Portfolio 7 

Examinations " 7 

Interview 6 

Competency statements - 5 

Supervisor 1 * statement 5 
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Conclusions . 



The following generalizations about standard-setting practices can 
be made based on the respondents 1 comments an'd attachments* 

1. A disappointingly high proportion of institutions award credit for 
experiential learning on the basis of experience rather than learning. 
For example, a handout for older students at a liberal arts college 
states: 

Some portfolios of prior learning reveal wide and rich 
experience but are quite limited in reflections; others show 
powerful reflective ability on scant experience. While 
neither of these would be awarded full credit (30 credits), 
they would probably receive a score in the 18-23 range. 

The evalua tor's guidelines at another institution state: 

The following guidelines roughly equate credit and 
practical on-the-job experiential learning: 18-24 months = 1 
credit hour; 25-36 months ■ 2 credit hours; 37-60 months .= 3 
credit hours; 60 months or more = 4 credit hours. 

2. The only standards or criteria reported by many institutions were 
those for determining whether the learning is college-level or credit- 
able. No other criteria or standards were described for converting the 
learning to credit after the learning has passed this first Screening. 
For example, a small liberal arts college's "Criteria for Creditable 
Learning" states: 

The College will award credit on the basis of the 
following factors: 

Learning: Credit for learning outcomes or compe- 
tencies, not experience. 

Precedent: This college has given credit in this 

area. 

Non-Routine: Learning can be distinguished from 
what everyone gains through standard life experience, 
or that gained before college entry. 



15 



9 



Independent from Credits Previously Earned: Learn- 
ing must not overlap with credits earned at other 
postsecondary institutions. Where learning is sequential, 
the student differentiates between levels of learning. 

Theoretical and Practical : Student should under- 
stand and articulate theory, history, principles and 
concepts of subject and be able to apply the knowledge 
in different situations. 

Applicable: Learning can be applied outside of 
specific situation in which it was acquired. 

- Demonstrable: Relationship evident between learn- 
ing and major educational goals. 

♦Verifiable: Learning can be demonstrated to an 
expert who thett evaluates level and quality of that 
learning. * 

3.^ Some institutions rely excessively on expert judgment of faculty 
members and fail to develop a consensus about standards and what 
constitutes adequate evidence of learning. For example, a small liberal 
ajrts college stated in its survey response: 

We have not established guidelines for formal standards. 
At this stage we prefer to rely upon the carefully exercised 
judgment of experienced academics. ... We keep in mind our 
institution's liberal arts identification and the standards 
we employed as classroom teachers and try to maintain 
comparable levels of expectation. 

If this sounds somewhat subjective, so be it. We have'" 
not yet found or conceived objective standards that we would 
prefer to the considered judgment of experienced professionals. 

A state university stated in its survey response: 

We have no written guidelines for faculty assessment as 
such, but there is a definite procedure which the faculty 
members follow in making their assessments. 

A student generally requests credit for specific courses 
from the university's undergraduate catalog. If fche student 
- can document that he has substantially met the course 
requirements as presented in the course description in the 
catalog, then he is eligible to request credit for that 
course. 
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If the professor basically agrees with the student's 
statement of competency, as supported by the documentation, 
then the request should be granted. 

4. In general, assessment application procedures provided in institu- 
tions 1 assessment program literature clearly tell the student what he 
or she must do. Little is conveyed to the applicant, however, about the 
assessment process and what might be expected of the assessor. Here is 
a typical set of instructions frcn a portfolio preparation workbook: 

If you are requesting credit for more than one course, 
please note the following: 

1. For each course, you will be required to provide a 
copy of the title page and a copy of the complete credit 
request list. 

2. A separate narrative description index and documen- 
tation section must be provided for each course* Submit your 
completed portfolio to the Director of Assessment and 
Evaluation. From there^ it will go to the appropriate academic 
department for assessment. 

The faculty of each department have determined which 
courses are assessable and how they are to be assessed. The 
faculty members themselves do the assessment of your portfolio 
and decide on whether or not to award credit. You are 
notified of the results through the Director of Assessment and 
Evaluation Office, which also notifies the records department 
so that your credit can be recorded on your transcript. 
Credit will be recorded using the appropriate course prefix 
and number and will be labeled , as assessment credit. No 
letter grades are assigned to assessed courses. 

5. Nearly all the responding institutions use CL2P (the College Level 
Examination Program) as a method of awarding credit for nonsponsored 
learning. Indeed, some institutions assess experiential learning by 
only using procedures recommended by CLEP and faculty agreement 
as the bases for awarding credit. 

6. Competency-based institutions or assessment programs have a sounder, 
more explicit basis for setting standards, which facilitates the 
conversion of experiential learning outcomes into academic recognition. 
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An assessment standard that relates a learning outcome to the content of 
a nutrition course states: 

Given a list of vitamins the student should be able to 
describe the chemical properties of, the lack of, the sources 
of, and the deficiency symptoms and diseases of the vitamins 
known to man. 

7. Without exception, standard setting for sponsored experiential 
learning is on firmer ground than standard setting for * nonsponsored 
learning, primarily because the learning outcomes, ^^jectives, and 
competencies for sponsored experiential learning are" often explicitly 
stated and agreed on at the outset of the learning activity, and the 
learning experiences are planned so that they directly relate to the 
outcomes. 




HOW SHOULD STANDARDS BE SET? 



The first question to ask about standard setting in assessment is 
whether experience, learning, or a combination of both is being assessed . 
In a traditional setting, credit is awarded for both* A student must 
be in the classroom a certain number of hours - experience measured by 
an attendance record - and must also demonstrate learning by means such 
as class participation, writing term papers, and performance on tests* 
In sponsored experiential learning credit is awarded for experience only 
(such as working in an approved setting) or for experience plus^^ 
demonstration of learning (such as meeting a specific job standard, 
keeping a diary, or participating in a seminar). ' The first rule in 
nonsponsored experiential learning, as represented in CAEL handbooks 
such as Forrest (1977) and Knapp (1977), is to award credit only on the 
basis of demonstrated learning and not on the basis of experience alone. 

A number of other questions may be subsumed under the more general 
question: Is this application for credit through experiential learning 
eligible for further processing? They include: Is the learning college 
level? Is the learning current enough? Would the credit, if granted, 
duplicate other credit the student has already earned? Wduld the credit, 
if granted, fit into the student's degree program; that is, can the 
student use credits in this subject area toward a degree? Does the 
learning represent more credits than a student is allowed to earn 
through assessment of experiential learning? 

These questions, although important to a college when it assesses 
experiential learning, are not central to standard setting. They are 
separate from the principal question: Is the learning worthy of credit 
at this institution, and if so, how much? 

For, if a student is to receive credit for experiential learning, 
it must be compared to some standard to determine its adequacy. Not 
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every instance of demonstrated learning is worthy of credit. How 
standards should be chosen is the focus of the rest of this report. It 
provides a framework for understanding the setting of staadards, reports 
the results of the authors' literature search, discusser principles of 
good practice in standard setting, and provides guidelines for deter- 
mining whether standards are adequate. 



Limitations of the Credit Hour 



The United States Bureau of Standards keeps a bar one meter long 
.under constant temperature and humidity in a fireproof vault. In theory^ 
at least, bars of other lengths may be "compared to it. The Bureau also 
has a kilogram weight comparable for comparison with other weights. It 
does not 'iave a semester-hour. 

The heart of the problem for those in higher education who want to 
equate an amount of learning with an amount of educational credit is 
chat although the semester-hour is thought of as the unit of educational 
credit, there is no standard semester-hour. 

Educators behave as if there were a standard s ernes ter -hour . In 
a traditional college setting credit hours are mainly used to define 
degree requirements. Credit hours are also used to determine how many 
courses a student may take at a time, how much tuition the student will 
pay, how many courses a faculty member must teach, how much money each 
department in the college should receive, etc. Thus the credit hour, 
standard or not, performs many administrative and economic functions. 

One of these functions, defining degree requirements, provides a 
good example to look at in more detail. Suppose the degree requirements 
at a. particular college for a Bachelor of Arts degree are 30 semester 
hours in each of four broad categories : humanities, social sciences, 
natural sciences, and frjee elect ives. In addition, the college may 
have lofty goals, such as: 
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• To provide tHe' citizenry with skills demanded by the continued 
change of civilization and the -development of a free society 

• To foster critical understanding and creative effort 

• To develop the student's philosophy of life and moral and spiritual 
values and to deepen his or her sense of responsibility toward the 
general welfare 

• To develop self -identity and self-conf idence and a sense of 
commitment to values, beliefs, and goals that are personally 
meaningful 

0 

In translating its goals into semester-hour requirements in various 
fields, the college' is assuming that (1) different students receiving 
thtee semester hours of credit in English literature from different 
instructors have made the same amount of progress toward attaining the 
goals of the college, and (2) a student receiving three semester hours 
of credit in English literature has made the same amount of progress 

toward attaining the goals of the college as a student receiving three 

» 

semester hours of credit in dance, French, or philosophy* 

■« 

These assumptions are questionable. In general, instructors teach- 
ing, the "same" English literature course will have their own course 
objectives, material to be covered, and examinations* Furthermore, they 
will have their own schemes for weighing diverse information (examination 
grades, term papers, class participation, attendance, student growth from 
beginning £o end 6f semester, etc.) to determine who should receive 
credit. When different courses in English literature are considered, as 
well as courses in other subject areas, the meaning of a credit hour 
becomes even murkier. In the traditional setting the credit hour not 
only has many functions, but it performs them imperfectly. 

THe aim of assessing experiential learning is to award academic 
credit for learning that took place outside the classroom, with the 
credit awarded being interchangeable with credit awarded for traditional 

classroom study. .However, the credit hour as a unit of educational 

* 

credit has majort limitations. The assessment of experiential learning, 
both sponsored and nonsponsored, often highlights these limitations and 
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makes educators more aware of the questionable assumptions that underlie 
the use of the credit hour. As one survey respondent from an institution 
that does not assess experiential learning wrote: "Although we do not 
give credit for learning outside the classroom, your survey ±\ eludes 
interesting questions for classroom learning 



Evidence of Learning 



Another complication in setting standards is that students may 
offer a wide variety of kinds of evidence to support learning claims. 
A standard may make sense for one kind of evidence and not be meaningful 
for others* 

The four basic categories of direct evidence of experiential 
learning include: (1) a written test, such as a comprehensive end-of- 
semester examination; (2) a performance test or observation, in which 
the student demonstrates knowledge or skills by doing , as in playing 
the piano or directing a small-group discussion; (3) an interview, a 
directed conversation through which the assessor determines the extent 
of the student's knowledge; and (4) a product evaluation, in which the 
assessor evaluates the student's previous work (e.g., drawings, articles, 
photographs). Indirect evidence of experiential learning includes 
portfolios and professional certification. These categories of evidence 
are shown in Figure 2. 

In setting standards, the first category of direct evidence, the 
written examination, is qualitatively different from the other three 
categories because comparable information is obtained from each student. 
Each student answers the same set of questions, and the resulting score 
can be compared . to the score of other students or to a predetermined 
level. 
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Figure 2. Ways of Equating Experiential Learning to Academic Credit x 
Kinds of Evidence Questions to Ask Alternatives for Setting Standards 
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This comparability is theoretically possible with each of the other 

three categories of direct evidence* For example, comparability might 

be achieved through a performance test in which each pianist plays the 

same pieces; through an interview in which each student is asked the 

same questions; or through a product evaluation in which each student 

sketches the same model, writes an article on the same topic, or 

photographs the same objects. But rarely, if ever, does a college 

< 

require this type of comparability. 

Because the written examination provides directly comparable 
student scores in an objective, inexpensive manner, it has received the 
most attention from educational psychologists and psychometricians who 
work in areas other than the assessment of experiential learning. 
Therefore, a literature search was conducted to determine whether the 
literature on setting standards for written examinations contains 
material pertinent to assessing experiential learning by examination. 

i \ 

The data base for the literature search included the ERIC 
Clearinghouse Dissertation Abstract? , and P^yoholog ical Abs tracts . 
Descriptors related to the topic \- such a&V Wademic standards, 
experiential learning, and standar^ setting - werfe used in the search. 
The search yielded several papers ^hat addressed problems of standards 
in education, minimum competency testing, academic Standards, and the 
setting of cutting scores for standardized tests. Umong the titles 
resulting from the search were: Report of the Coimpi$sion on Academic 
Standards , Proceedings of National Conference on MjLnamum Competency 
Testing , and Setting Standards for Basic Skills Reading \Msessment . 

\ \ 

In general, the papers focused either on the general issue of 
lowered quality of standards in education or on approaches to, setting 
minimum performance standards for standardized tests. Thus,^the problems 
and issues discussed in most of the papers related to thifr study only 
tangentially. Those that addressed the topic of this study\ are listed 
in. the references. 
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Fixed Quo ta and Quota-Free Decisions 

The literature suggests that the first question to be considered 
is why a -cutting score should be set. The goal may be" to pass a certain 
number of test applicants to fill a predetermined number of places or to 
pass all applicants who are' qualified. Shepard (1976) refers to this as 

the distinction between fixed-quota and quota-free decisions. 

• * 

Here is an example of a fixed-xjuota decision: Thers are places for 
120 new students, so a cutting score is set that passes 120 test 
candidates. Possible complications with such a fixed-quota decision 
include: 

• Based on past experience, only 6g percent of the test candi- 
dates that pass will enroll at the school; therefore, a cutting 
score is set that will pass -200 candidates (rather (ban 120). 

• As the population of applicants shifts from year to year, the 
cutting score shifts and standards drift. 

^ • As the number of available places changes from year to year, 
the cutting score shifts and standards drift. 

In another example of a fixed-quota decision, marketplace consid- 
eratlons lead a professional' association to determine that 250 more 
people should * be certified this year. The association sets a cuttirg 
score that passes the highest scoring 250 candidates. To the extent 
that the test actually measures it, th^ professional competence of the 
people certified each year will depend on the professional association's 
perception of marketplace considerations. 

Here is an example of a quota-free decision: A driver's license is 
to be issued to each person who can demonstrate that he or she caa safely 
operate an automobile. Test* construction and automotive safety experts 
design a test and choose a cutting score. Possible complications with 
such a quota-free decision include: 
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• If the . actual accident rate exceeds a tolerable level and 
testing standards are thought to be at fault (in contrast to, 
for exauple, poor automotive de*i^n), then ^he_cut ting score will 
be raised to reduce the accident raLe and, iiTaddition, the nature 
of the test may be altered* 

• If the driving test is a barrier to "too many people receiving a 
license, so* that they cannot take part in the complex social and 
economic activities of the society, the cutting score will be 
lowered to eliminate the barrier and Che nature of the test may 
be altered* 



Here is another example of a quota-free decision: f A basic tenet of 
CAEL is that * adults should receive credit for college-level knowledge 
that they can demonstrate. Accordingly, a college sets a cutting score 
on a CLEP examination based on faculty judgment after comparing the 
examination to courses taught at the college* The college award* credit 



to those whose sopre equals or exceeds the cutting score* Possible 
complications with such a quota-free decision include: 

• The college finds that students who receive credit for an 
introductory course through CLEP do not have sufficient background 
for a more advanced course for which the introductory course is a 
prerequisite* The college raises the cutting score. • 

• The college finds that students whc receive credit for an 
introductory course through CLEr* are superbly qualified for, a 
more advanced course for which the introductorv course is a 
prerequisite. So many students, /however," are earning credits 
through CLEP, that a number of tenured faculty members have too 
few students^ to teach. .The college raises the cutting score. 

• The college fiuds that it is not getting its share of * the return- 
ing-adult-student market. Since nearby colleges have lower 
cutting scores for CLEP examinations, the college ' lowers its 
cutting 8 cores* 

These examples illustrate how social policy and economic considera- 
tions affect the setting of cutting scores, in both nonacademic and* 
academic situations arid how what appears to be a quota-free situation may 
implicitly be a fixed-quota situation. 
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Setting Cut-Off Scores for Examinations 



When awarding credit for experiential learning through examination 
is regarded as a quota-free situation, a choice must still be made about 
how to set the cut-off score. 

Norm-referenced measurement * Using an examination to award credit 
by comparing a student's score to the scores of others is a form of norm- 
referenced measurement. The cutting-score problem (and the standards 
problem) then becomes: What proportion of scores should the given 
student's score exceed for the student to be awarded credit? 

CLEP offers an interesting example of the resolution of this 
problem* CLEP subject examinations correspond to specific college 
courses widely taught across the country; students may take these 
examinations without taking the corresponding courses and request credit 
from their respective colleges on the basis of their 3cores. There are" 
three major approaches colleges have taken in determining acceptable or 
parsing scores for CLEP Subject Examinations at their institutions: 

1* A reference group of students has completed both relevant courses 
and the corresponding. CLEP. examinations , and the mean examination scores 
of students receiving an A course grade, a B course grade, etc., have 
been computed. The Commission on Educational Credit and Credentials of 
the American Council on Education has recommended that colleges award 
credit at the mean C level for each examination. A large group of 
colleges has accepted this recommendation. 

2. § Another group of colleges has chosen to award credit to students 
whose scores exceed 50 percent of the scores in the reference group 

9 f 

mentioned above. 

3. Some colleges have chosen to set their own passing scores for each 
examination. 

The first two of these approaches provide a uniformity of standards in 
one area (credit by examination) among institutions that otherwise may 
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be quite diverse in their academic standards for admission and in their 
degree requirements. 

Criterion-referenced measurement . An alternate approach is to use 
an examination to award credit by comparing a student's score to a 
predetermined standard, a form of criterion-referenced measurement. The 
cutting score problem (and the standards problem) then becomes: How 
should the standard be determined? 

One approach is for several faculty members who teach the "same" 
course to develop an examination for that course and then discuss with 
each other what minimal passing score will demonstrate mastery of the 
course content. However, a danger in this approach is that often faculty 
members will base their judgments on a "conventional" passing score ("I 
think 65 percent and above should be considered passing") without 
considering in detail the actual content of the examination. This way 
of setting a passing score is particularly ; inappropriate when the test 
does not cover the specific course content taught by any of the faculty 
members, but instead represents a consensus of an idealized course that 
no one teaches. For example, during the development of an art history 
examination one faculty member will keep in mind the specific works of 
art he or she usually discusses in class, and another faculty member 
^will do the same thing, but have in mind different works of art. When 
these faculty members agree on the content and the cut-off score for an 
examination for experientially trained adults who have not had either of 
their courses, a 65 percent cut-off score is likely to be inappropriately 
high or low, depending on the fit between what the faculty members choose 
for content and the students 1 experiences. 

There are other methods for setting cut-off secures that force 
faculty members to directly confront the actual content of the examina- 
tion on a question-by-question basis, for example, Nedelsky (1954) -and 
Angoff (1971). Although such methods provide a veneer of psychometric 
sophistication, there remains the problem of the fit* between test content 
and student knowledge. 
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Setting Standards for Other Types of Assessment 




Because evidence for experiential^ learning is most often not 
comparable from student to student, other approaches for setting stand- 
ards in the evaluation of experiential learning must be used. Students 
requesting credit in journalism/ for example, may present portfolios 
of articles they wrote that cover different news stories. The precision 
of the psychometric approaches for setting cut-off scores for examina- 
tions that provide comparable evidence of learning is not possible here. 
Tb* * are basically two choices. The first is to impose comparability 
on the criteria for reviewing the evidence of learning that students 
present. The second is to judge the adequacy of the student's evidence 
of learning against well-defined dimensions. 

Imposing comparability. While students-as-journalistsr may have 
covered different stories, there is probably a set of criteria that can 
be applied to the rather diverse articles they have produced. Suppose, 
for example, a group of journalism professors agreed that student* 
passing Journalism 104 should be able to write articles that were 
satisfactory in five respects: accurate, major points emphasized, 
interestingly written, grammatically written, and stylistically appro- 
priate. The standards for judging diverse student products would the'n 
be how satisfactory^ each was on these five criteria. 

How might the adequacy of the standards themselves be judged? 
Here are three major approaches: 

1. Would a larger group of journalism professors accept these five 
criteria? 

2. How well would journalism professors, using these five criteria, 
agree among themselves in independently rating the same student 
articles? 

3. How well would the students recommended for credit in Journalism 104 
(rated satisfactory on all five criteria) later perform in Journalism 
105 (for which the prerequisite is Journalism 104)? 
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These three ways of judging the adequacy of the standards may, of 
course, be generalized to assessments of experiential learning in other 
subject a^eas: 

1* Consensus among experts as to criteria 

2* Agreement among experts in independent judgments 

3* Future performance of students who are awarded or denied credit 
In addition, these' three ways of judging the adequacy of standards are 
roughJLy analogous to content validity , inter-rater reliability , and 
predictive validity , in more conventional psychometric terms* 

Well-defined dimensions * Assessments of experiential learning 
based on interviews, product evaluation, and other portfolio evidence 
and -information 'that is not comparable from student to student are 
problematic in the setting of standards* One solution is used at Thomas 
A* Edison State College, which requires its faculty assessors to consider $ 
portfolio evidence of student learning along four independent dimensions: 

1* Directness * A copy of an article the student has written is direct 
evidence of the student's writing ability* The fact that the student 
was employed as a technical writer is indirect evidence of the student's 
^writing ability* In general, direct evidence is preferable to indirect 
evidence • 

2. Authenticity * Did the student write the article himself jQt hers^f? 
Did the student actually work as a technical writer? 

3. Breadth * Is a single article a student has written a sufficient 
basis to recommend awarding credit? What a1)Out three articles on the 
same topic? Three articles,- each on a different topic? Is a student's 
employment as a tec hnical — writer — for one month a sufficient basis to 

jonnJCna^awarding credit? For six nunths? For one esployer? For 

several employers? r 

i ■ 

4. Quality * How good are the articles in the portfolio? How well did the 
student work as a technical writer? 

On each dimension, there is, of course, a large judgmental factor* 
The primary mechanism for maintaining adequate standards would be, once 
again, to see how well faculty judgments, independently made, agree with 
each other. ^ 
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, Evaluating the Adequacy of Standards 



Clearly, standard setting is a highly judgmental process, with 
faculty or institutionally appointed experts at the center of the 
process. Furthermore, consensus of experts is the primary mechanism for 
maintaining adequate standards. 

Guidelines are required to develop and maintain standards, Premer 
(1976) has suggested the following criteria for judging the adequacy of 
standards: 

' !• Comparability and realism. Standards should be developed so that 
there is comparability of requirements for experiential learning and 
traditional classroom learning. Standards must be set at high but 
realistic levels, H , — 

2» Definitions of adequate learning . Standards should be accompanied 
by clear definitions of what constitutes adequate learning, 

3» Consistency and reliability , Stand^nls^should be applied consistent- 
ly within programs, across programs , jrfid over time, 

*• Acceptability and Transferability , Standards should be backed fully 

• by- the college to increase the likelihood that the resulting credit will 
be acceptable to other institutions, 

5. Adequacy of preparation and feedback. Standards should be adminis- 
tered with an adequate student advisory and guidance system so that 
standards are not accompanied by high failure rates, 

6, Student satisfaction. Standards should be administered so that the 
students involved feel they have been well served, 

7» Management standards . Standards should be developed using good 
management standards or procedures, 

8» KtKics and Responsibility , Standards should be developed by a process 
that not only gives majot attention to the needs of students but also 
recognizes the institution's responsibility to societal needs, 

Sound standards are central to the credibility and acceptance of 
credit for experiential learning. When carefully developed, appropri- 
ately applied, and closely monitored, standards assure *the student, the 
school, and the public that credit awarded is credit awarded for college- 
level learning, and that the amount of credit awarded is appropriate and 

• fair. 
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